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I. INTRODUCTION 


As the computing needs of an organization expand, the 
decision to add to, or replace, existing computer svstems is 
important and may have long term effects on an organization. 
The impact can be financial, operational or both. The 
choice of computer systems, hardware and software will be a 
major factor in defining the information processing 
capabilities of an organization. The methods for selecting 
computer systems are varied and have evolved over the past 
20 years. 

Computer systems have become less expensive and, at the 
same time, significantly more powerful. Advances in data 
communication technology have led to the increased use of 
distributed data processing (DDP) and decentralized computer 
systems. In order to effectively use DDP and decentralized 
computing, computer systems throughout an organization must 
be compatible with each other. Data and files have to be 
transferable and useable on all systems throughout an 
organization. 

Once a decision is made to acquire a new computer 
system, an organization needs to ensure that its selection 


is the best choice to meet its present and future computing 


needs. If the selected computer system does not fulfill the 
organizational requirements there might not be another 
chance to make a better choice due to cost and time 
constraints. 

The emphasis of this thesis is on the evaluation and 
selection phases of a computer system acquisition process. 
The actual selection of a computer system is preceded by 
several procedural steps. The following is a generalization 
of those procedures: 


e Analyzing the requirements and computing needs of the 
organization. 


¢ Determining and defining the requirements for the 
computer system. 


¢ Sending the request for proposal (RFP) to qualified 
vendors. 


¢ Screening, evaluating, validating and comparing the 
proposals. 


¢ Selecting the computer system. 

From the multitude of computer systems, operating 
systems, software and peripherals, a user has to choose 
those products that best meet an organization’s needs. The 
difficult task facing those responsible for the selection is 
to determine by what standards competing systems are 


compared and evaluated. 


Traditional procedures have addressed the selection of 
individual computer systems. With DDP and decentralized 
computing becoming more prevalent, there is a need to apply 
new methods for selection. The concern is no longer the 
selection of a single system; it is the selection of a group 
of systems that work together and enable the sharing of 


files and date 


II. COMPUTER SELECTION PROCEDURES 


As long as there has been more than one computer system 
available to meet a user’s requirements, there has been a 
need to select the one that best fits the job. The problem 
is how to make that selection. There have been a number of 
models and methcds proposed anda used to select computer 
systems. The various methods used have concentrated on the 
individual stages in the selection process and on specific 
techniques for each stage. 

The selection process consists of several stages: 
analysis of needs, determination of requirements, request 
for proposals, evaluation of proposals and selection of a 
system. The first two steps are preliminary to the 
evaluation of the proposals, while the latter two are often 
blurred into a single step: evaluate the candidates and pick 
the best. 

The techniques for the evaluation of computer systems 
generally are either simple or sophisticated. The simple 
methodologies are basically intuitive, unsupported by any 
theory or lacking a systemized approach. The sophisticated 
methods employ an analysis of vendors’ proposals with a 


ranking of the proposed computer systems. Following this 


ranking, a hardware performance and capability analysis is 
performed on the most desirable alternatives. If the system 
that was ranked first fails to perform adequately then that 
system is rejected and analysis is carried out on the next 
highest ranked system. 

The most commonly used computer system evaluation 
methods are: 

Weighted Scoring. This commonly used approach attempts to 
overcome evaluation problems by combining objectivity with 
consideration of apparently non-quantifiable factors. 
Relative weights are pre-assigned to all system items 
considered important. Each competent system is subjectively 
rated with respect to each selected system item and then the 
overall score is computed for each vendor. The rating is 
subjective: from "no good" to "entirely adequate”. The 
system with the highest total score is considered most 
desirable. (Borovits and Zviran, 1987) 

The advantage of this method is that it is simple. It 
also enables one to perform sensitivity analysis of the 
results for changes in the importance weights of the 
attributes. Its disadvantage is that it is not normative, 
i.e. it is not based on a system of axioms expressing 


rational behavior rules expected of a decision maker. Thus, 


it does not require an examination of assumptions of 
independence in the attributes and the absence of such an 
examination may result in deviations. (Shoval and Lugasi, 
1987) The following example demonstrates this: 

Two proposals, A and B, are examined according to two 
independent attributes, "vendor support" and "hardware 
performance". If A and B receive the same score in the 
evaluation and if "vendor support" is evaluated as entirely 
adequate in proposal A and no good in proposal B while 
“hardware performance" is entirely adequate in B and no good 
in A, then the "hardware performance" of B may never be 
available because of its no good "vendor support". 

(Tiammreck, 1973) 

This method does not allow for an examination of 
consistency of the evaluators. Due to its nature, the 
method does not consider risk and uncertainty. It should be 
noted that granting scores in every attribute is in itself a 
difficult task and influenced by subjective considerations. 
(Shoval and Lugasi, 1987) 

Multi-Attribute Utility Model. Shoval and Lugasi (1987) use 
of Keeny’s multi-attribute utility model allows an 


evaluation of the utility function of the attributes and the 


calculations of their weights. It differs from other models 
in that it considers risk and uncertainty. 

The application of the model requires both utility and 
preference independence. Utility independence claims that 
for attributes X1...Xn the utility of attribute Xi does not 
depend on the remaining attributes. Preference independence 
claims that preference between every pair of attributes does 
not depend on the fixed level of remaining attributes. 

There are two variants, which differ in the way they 
refer to risk, the additive model and the multiplicative 
model. The application of either depends on a decision 
maker’s attitude towards risk. 

In order to evaluate the utility function of the 
attributes and to calculate the weights/constants of the 
attributes, a gambling technique is used, a utility function 
of the Von-Neuman-Morgenstern type. 

Evaluation of the decision maker’s utility function is 
based on the axioms of transitivity and continuity. If 
these axioms are satisfied, it is possible to form a utility 
function for the decision maker for each attribute. 

The multi-attribute utility model is normative, since it 
forces a decision maker to accept a set of axioms which 


express preferences and it also requires an examination of 


independence assumptions before the model is applied. The 
model also enables one to make sensitivity analyses. 
Compared with other models this model is more difficult for 
the decision maker to understand and apply, since 
determining a probability at which a decision maker is 
indifferent between alternatives is not always clear and 
acceptable. (Shoval and Lugasi, 1987) 

Cost-Effectiveness Ratio. This is simply a sub-category of 
the weighted scoring technique, where the cost of the entire 
computer system is divided by the sum of its points. The 
system having the highest total score is considered the most 
desirable. (Borovits and Zviran, 1987) 

Cost-Value. This approach uses cost as the sole measure. 
Cost and benefits are assigned to the desirable features of 
each proposed system. The sum of these values is then 
subtracted from the total cost of the system. The result 
represents the cost of the system to meet to meet the 
mandatory requirements. The system with the lowest cost is 
the most desirable. (Timmreck, 1973) 

Requirement-Costing Technique. Here the estimated cost- 
savings benefits are assigned to a set of desirable 
features. Those not part of the proposed system are then 


added to the total cost of the system. In addition, if a 


desirable feature is offered at an additional cost that is 
less than the estimated benefit, the incremental cost is 
added to the system cost. The system with the lowest 
resulting cost is considered the most desirable. 

(Borovits, 1984) 

Dynamic Approach. This approach employs a projected work 
load growth trend and cost-effectiveness ratios of each 
vendor’s proposed product line. The underlying assumption 
is that it would be feasible to install a system for as 
little as one year, but only if it will be replaced by a 
compatible system from the same manufacturer’s product line 
and replacement accords with the dictates of work load. 
Using the projected work load trend and measurement of 
system capacity, a schedule of replacement is worked out. 
The cost of each system (as determined by one of the methods 
described above) over the period of its employment is then 
discounted with respect to an estimate of cost of capital 
and its expected future life. The product line with the 
lowest discounted present cost is the most desirable. (Ein- 
Dor, 91977) 

Present Value Analysis. The objective here is to determine 
which system will cost the least and benefit the 


organization most. The net present value of the proposed 


system should be determined by discounting all cash flows 
associated with it, using an organization’s cost of capital 
as the discount rate. The benefits projected by the 
proposed new system are the net cash inflows. The system 
with highest net present value is the most desirable. 
(Borovits and Zviran, 1987) 

Efficient-Frontier Model. In this model, the preferred 
system is that which is dominant in all the attributes 
considered according to a decision maker’s preference. 

This method enables one to screen out alternatives that 
are inferior in all attributes or are identical in some and 
inferior in at least one. The alternatives remaining after 
this screening form an "efficient frontier". There remains 
the problem of selecting a preferred alternative; this 
requires other methods. Therefore, this model is not 
sufficient. It is applicable, at most, in performing 
initial screening. (Shoval and Lugasi, 1987) 
Lexicographical Ordering. In this model, alternatives are 
rank-ordered according to a dominant attribute, the most 
important one, e.g. CPU power. Clearly, this is possible 
only if a dominant attribute exists and cannot be traded off 


with others. This model can, at most, help in the initial 
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screening of alternatives which are deficient in an 
important attribute but it cannot choose the best. (Ahituv 


and Neumann, 1986) 
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III. COMPUTER-FAMILY SELECTION 


The selection procedures being discussed focus on the 
selection of a group of systems that will work together, 
sharing files and data. This group of systems has been 
called a computer-family and defined as: 


Computers of the same type, consisting of several models 
from the same manufacturer’s product line, ranging from 
microcomputer to mainframe, with full compatibility in 
the operating system and the system's software, to 
enable transfer of application software from one family 
member to another without change. 

(Borovits and Zviran, 1987) 


An example of a computer-family is Digital Equipment 
Corporation’s (DEC) VAX 8840 mainframe computer, VAX 6220 
minicomputer and Microvax 3600 super-microcomputer. Another 
example consists of a Prime 6650 mainframe computer, a Prime 
4450 minicomputer and a Model 4050 super-microcomputer. 

Selecting a computer-family is a more complex process 
than selecting a computer system. The process will have a 
lasting effect on an organization because there will be one 
computer-family from which current and future hardware 
acquisitions will be made. This will help ensure uniformity 
and compatibility in information processing throughout an 


organization. 
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For DDP and decentralized computing the compatibility 
obtained in hardware and software will preclude the systems 
integration problems encountered while using mismatched 
systems. The benefits of system-wide compatibility are 
exemplified by the ability to transfer application software 
from one family member to another using a common operating 
system. 

A number of methods are used for computer system 
selection. Some of the concepts presented in the existing 
methodologies form the basis for a methodology to deal with 
computer-family selection. The procedures presented here 
are a ten-step generic evaluation and selection methodology. 
(Borovits and Zviran, 1987) 

Figure 1 portrays a work flow diagram of this process. 
The work flow diagram highlights the processes that occur 
after an analysis of an organization takes place and 
systems’ requirements document are developed. 

The stages in this process are: 

Step 1. Identification of possible vendors and 
manufacturers: After studying the requirements document it 
may be determined which manufactures or vendors may be able 


to provide products that will meet an organizational needs. 
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Step 1. Identification of possible 
vendors and manufacturers. 


Step 2. Primary elimination of 
irrelevant candidates. 


Step 3. Determination of 
mandatory requirements. 


Step 4. Examination of vendors’ compliance 
with mandatory requirements. 


Step 5. Setting quantitative and qualitative 
criteria and respective weighting scales. 


Step 6. Writing the RFP to be addressed 
to selected vendors. 
a 


’ 


Step 7. Receiving, comparing and analyzing bids. 


Step 8. Concluding final list of vendors. 


Step 9. Performance of hardware 
and software benchmarks. 


Step 10. Drawing final conclusions 
and selection of best computer-family. 





Figure 1. Selecting a computer-family: a work flow 
diagram. (Borovits and Zviran, 1987) 
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A list of possible vendors can be generated and used in 


future stages. 


Step 2. Preliminary elimination of irrelevant 
candidates: Further study of an initial list of 
manufacturers and vendors should reveal apparently 
unsuitable candidates. They should be pared from the 
original list to produce a manageable list of potential 
(relevant) vendors. Reasons for elimination are for 
unsuitability in meeting requirements or because they do not 


have a complete computer-family. 


Step 3. Determination of mandatory requirements: From 
the systems requirement document and from organizational 
policy certain features and characteristics of a computer- 
family are identified as prerequisites for further 
consideration. These are criteria that must be met by any 


potential computer-family. 


Step 4. Examination of vendor’s compliance with 
mandatory requirements: On the basis of mandatory 
requirements, a questionnaire is formulated and addressed to 
all relevant vendors. Responses to this questionnaire 


provide a selection committee with basic information about 
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proposed computer-families and each product line. The 
responses are screened to determine if they meet mandatory 
requirements. The vendors remaining after this elimination 
procedure constitute a mailing list for requests for 


proposals (RFP). 


Step 5. Setting quantitative and qualitative criteria 
and respective weighting-scales: The variability of the 
capabilities and performance characteristics of each element 
(hardware, software, service) of a computer system is very 
large, and the number of permutations is considerable. A 
properly constituted set of quantitative and qualitative 
criteria is critical for the acquisition of a satisfactory 
computer-family. Furthermore, vendors are expected to 
propose not a single computer system but rather a wide 
product line of systems working together. This raises a 
problem of evaluating and comparing proposals. Borovits and 
Zviran (1987) suggested that the quantitative selection 
criteria be applied to at least the following five issues, 
each of which has its own partial list of guidelines. 


[Eigen 
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HARDWARE (for each model within the family) 


* memory size (minimum, maximum, units of expansion) 

x disks (number of drives, minimum and maximum capacity) 
* tapes (maximum number of drives, density) 

* diskette readers (diskette size, density) 

* terminals 

* printers 

* backup features 

* miscellaneous 


SOFTWARE (for the entire family) 

* system software (uniformity of operating system, compilers, 
utilities, etc., throughout the family) 

* programming languages (compilers and interperters, options 
and extensions, adherence to standards) 

x development tools (existence of application generators, re- 
port generators, full screen editor, debugging aids, macro 
facilities, etc.) 

x additional software (electronic-spreadsheet, word-processor, 
electronic-mail, DBMS, etc.) 

* ability to transfer software within the family (source mod- 
ules, load modules, procedures) without change or conver- 
sion 

COMMUNICATION 

* communication protocols 

* existence of LAN, possibilty of connection to LAN 

* possibilty to transfer files, programs and procedures through 
communication 

* network transfer rate 

CONVERSION 

* possibilty to convert application software (programs and 
procedures) from present system to proposed computer- 
family 

ENVIRONMENT 

* environment requirements (such as airconditioning, temper- 
ature, humidity, etc.) 


Figure 2. List of criteria. (Borovits and Zviran, 1987) 
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Certainly not less important, though non-quantifiable, 
are qualitative criteria such as: 


e how widespread is the use of the proposed computer- 
family 


* vendor support, such as training, maintenance philosophy 
and practice 


* software houses specializing in the proposed computer- 
family 


* current users’ opinions and vendor’s reputation 

Once a detailed and complete set of data relating to all 
criteria is collected, weights are assigned to all items, 
both qualitative and quantitative, indicating their relative 
importance. These weights are fundamental to the evaluation 
process, explicitly indicating that the objective of the 
evaluation process is to ensure the selection of a computer- 
family that best fits the needs (present and future) of an 


organization. 


Step 6. Writing the RFP to be addressed to selected 
vendors: The primary advertising medium is the RFP, which 
is issued by a procuring organization and contains the 
system requirements. Typically, the RFP will include a 
summary list of specific requirements according to which the 


vendors will be asked to write their proposals, describing 
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how they will meet each requirement. The RFP should consist 
of two major parts, the first relating to each model within 
the computer-family and the second to the computer-family as 
a uniform entity. The second part is more general in nature 
and concentrates upon such issues as system software, 
conversion of present application software to the new 
computer-family, environment, etc. The RFP should be 
addressed to those vendors satisfying mandatory 


requirements. 


Step 7. Receiving, comparing and analyzing bids: The 
problem of analyzing and comparing bids is particularly 
important, since it is more than possible that none of the 
computer-families under consideration may exhibit a clear 
dominance over others. This stage is, therefore, 
fundamental to the interim evaluation and selection process 
and should be accorded an appropriate measure of time and 
effort. It is required that the bids be submitted in 
writing and it is expected that they will conform to the 
style indicated in the RFP, so that the selection process 
will not be affected by style of expression and use of 


selling techniques. 


19 


The first step in the process of analyzing bids to 


subjectively rate each competing system with respect to each 


evaluation criterion. A score is assigned to each item. 


The 


weighted score is then computed for each proposed 


system. The next stage is the most problematic since there 


are 


no methods or guidelines for comparing computer- 


families. The proposed method consists of the following 


steps: 


Five categories of computers are established: 
microcomputers (single station) 
super-microcomputers 
minicomputers 
super-minicomputers 


mainframe computers 


These categories represent differences in computing power 


and 


other major hardware characteristics. 


Relative weights are assigned to each category according 
to its importance for an organization. 


Each model within each computer-family is assigned to a 
category on the basis of criteria such as memory size, 

maximum number of terminals, maximum disk capacity and 

relative performance. 


For each criterion, a computer model, or models, is 


selected within each category, which has dominance over 
its competitors and the score of 100 is assigned to it. 
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All other models of the remaining computer-families are 
then respectively an subjectively scored on a scale of 
oO - 100. 

¢ The total score is calculated. The score attained for 
each criterion is the average weight calculated for all 
five categories. 


e A comparative table is drawn up [Fig. 3] giving the 
scores attained by each family for each criterion. 


¢ The final score achieved by each family is calculated. 


Step 8. Drawing up a final list of vendors: On the 
basis of the final scores attained by each computer-family a 
selection committee is able to disqualify irrelevant 
computer-families, and select up to three or four vendors 
most likely to succeed. These computer families are then 
further tested to ensure they have the proper capabilities 


and characteristics. 


| Criteria | Vendor A | Vendor B | Vendor C | 
Crit. 1 
Grit 2 
Grit. 3 


Figure 3. Outline of comparative table. 
(Borovits and Zviran, 1987) 
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Step 9. Benchmarks for performance of hardware and 
software: Study of the written proposals is sufficient for 
evaluating fulfillment of certain requirements. In such 
cases, benchmarking must be performed. A benchmark, in the 
context of this discussion, is a set of live tests designed 
to examine the computer systems (hardware and software) 
proposed in response to an RFP. Selection of criteria to be 
tested may be performed according to their importance for an 
organization using the relative weights already assigned. 
The purpose of these benchmarks is to verify a system’s 


characteristics before drawing final conclusions. 


Step 10. Final conclusions and selection of the best 
computer-family: After benchmarks have been performed and 
all essential characteristics of a proposed computer-family 
have been deemed satisfactory, a selection committee will 
review and reconsider the relevant scores assigned to each 
competing computer-family. 

Finally, a committee will pick the best as the one 
recommended to be an organization’s computer-family. The 
recommendations will then be submitted to an organization’s 


management for approval and adoption. A problem faced by 
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those involved in the selection process is how to compare 
criteria and how to prioritize them according to their 
importance to the decision making process. There will be a 
large number of criteria, some quantifiable and others non- 
quantifiable, whose importance to the selection process will 


be compared with each other. 
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IV. THE ANALYTICAL HIERARCHY PROCESS 


The methodology developed by Borovits and Zviran (1987) 
for computer-family selection discussed briefly the process 
of assigning relative weights to each category of computer 
in a family being evaluated (Step 5). The process continues 
with each criterion assigned a relative weight so that it 
can be evaluated and scored. In a case study by Borovits 
and Zviran (1987), the assigning of the relative weight and 
the actual scoring of computers-families was done 
subjectively. 

Using subjective weighting and scoring can reduce the 
overall effectiveness of the process. There is little 
precision and it may be difficult to achieve replication 
using subjective weighting and scoring. 

An objective weighting and scoring system can improve 
this process. An objective weighting system should be 
characterized by consistency in the assignment of weights. 
Consistency is a factor that can be computed and compared to 
a standard. The standard is somewhere less than perfect 


consistency and greater than intolerable inconsistency. If 
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a standard is exceeded, this indicates randomness in the 
assignment of values. Should this occur, the criteria 
should be re-evaluated. (Saaty, 1980) 

An objective method that can be adapted for the 
comparison of a large number of objects, as is the case in 
comparing computer-families, is the Analytical Hierarchy 
Process (AHP) developed by Saaty (1977). 

The AHP can properly assess the importance of a large 
number of interacting factors, develop priorities among the 
factors and choose a best alternative (Saaty, 1980). 
Seidmann and Arbel (1985) suggested an application of the 
AHP to the process of microcomputer selection. A large 
number of or attributes were used to compare microcomputers. 
The use of AHP allows the determination of both weights and 
scores for each attribute for each alternative, using 
Iatrices to perform pairwise comparisons between 
alternatives. Once weights and scores are obtained, the 
final score of each alternative is calculated according to a 
weighted scoring technique. 

In order to determine the weights of n attributes, a 
decision maker fills up a matrix with dimensions nxn in 


which a pairwise comparison is made between every two 


Zo 


attributes. (Fig. 4) Thus, in each cell (i,j) in the 
Matrix, a decision maker expresses the relative importance 
of attribute i with respect to attribute j. (It is only 
necessary to fill up half of the matrix since a,, is the 
reciprocal of a ,,.) Then, an eigenvector of the matrix is 
calculated for a maximum eigenvalue. The eigenvector is 
normalized so that the total sum of its elements is 1. The 


values of this eigenvector constitute the attribute weights. 


Politi- Ideolog- Techno- Eigen- 

Economic Social cal ical logical vector 
Economic al 7 &) 7, 3 45 
Social ab il ny A} 7 1/6 .09 
Political 1/3 3) il i, 1/2 pele 
Ideological a 1/7 1/7 dl 1/8 93 
Technological 1/3 6 2 8 1 ae 


Figure 4. Matrix of attributes. (Saaty, 1982) 


To determine the attribute scores for each alternative, 
a decision maker fills a matrix of pairwise comparisons 
between alternatives. Altogether, n matrices are filled 
(one for each attribute). The dimension of every matrix is 


the number of alternatives. Here, too, a normalized 
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eigenvector is calculated and the elements of each such 
vector express the scores of the alternative for each 
attribute. 

Pairwise comparisons are accomplished with a relative 
numerical scale. Saaty (1977), in agreement with G.A. 
Miller’s 1956 experiments proving individuals cannot 
simultaneously compare more than seven objects (plus or 
minus two) at the same time without being confused, chose a 
numerical scale of 1 - 9. Using this scale, 1 expresses 
identity between two objects being compared and 9 expresses 
an absolute preference of one over the other. [Fig. 5] 

The Eigenvector method enables one to examine a decision 
maker’s consistency, using appropriate measures (in essence, 
if a decision maker is consistent in a certain matrix, the 
maximum eigenvector of the matrix equals the order of that 
matrix). On the other hand, this method ignores 
independence between attributes. It also does not consider 
risk and uncertainty. Nor does it enable one to perform 
sensitivity analyses. (Shoval and Lugasi, 1987) 

The AHP removes the guess work from the assessment of 
the relative importance of various criteria. The AHP can 


assist in analyzing several variables simultaneously. A 


Pa) 


hierarchy is established so that sub-criteria relevant to a 


Main criteria can be prioritized and comparisons can be made 


of minute details. (Saaty, 1980) 


Intensity of 
Importance 


Definition 


Explanation 





2,4,6,8 


Recirocals 


Equal importance of one 
element over another 


Weak importance of one 
element over another 


Essential or strong impor- 
tance of one element over 
another 


Demonstrated importance of 


one element over another 


Absolute importance of one 
element over another 


Intermediate values between 
two adjacent judgments 


If activity i has one of the 
preceeding numbers assigned 


to it when compared with 
activity j, then j has the 
reciprocal value when com- 
pared with i 


Two elements contribute 
equally to the property 


Experience and judgment 
slighty favor one element 
over another 


Experience and judgment 
strongly favor one element 
over another 


An element is strongly 
favored and its dominance is 
demonstrated in practice 


The evidence favoring one 
element over another is of the 
the highest possible order of 
affirmation 


Comprimise is needed 
between two judgments 





Figure 5. Saaty’s comparison table. (Saaty, 1982) 
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V. PROPOSED METHODOLOGY 


Applying the AHP to Step 5 of Borovits and Zviran’s 
methodology (Fig. 1) makes the resolution of ranking and 
weighting alternatives less arbitrary. 

In selecting a computer-family, specific attributes, 
capabilities, and performance characteristics, collectively 
known as criteria, are used as the basis for comparison 
between the competing alternatives. In Step 5 of Borovits 
and Zviran’s methodology, criteria are identified, 
prioritized and weighted according to their significance and 
value in the selection process. This procedure yields a 
maximum value for each criterion. Using the maximum values 
for all of the criteria, the highest possible score for each 
computer-family is computed. The maximum attainable scores 
in Step 5 are used in Step 7. 

The AHP allows a decision maker to objectively create a 
prioritized and weighted list of criteria. At each level of 
the hierarchy, every criterion is compared to all the others 
in its group, on a one-to-one basis. Using the scale and 


descriptions from Figure 5, a score for each pairwise 
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comparison is obtained. These scores are inserted into a 
matrix to compute the weight of each criterion by Saaty’s 
eigenvalue method. The weight of each criterion is used 
with the weights of the criteria that are above it in the 
hierarchy to compute a maximum value. 

To facilitate the incorporation of the AHP into Step 5 
of Borovits and Zviran’s methodology (Fig. 1), setting 
quantitative and qualitative criteria and respective 
weighting scales, the prioritization of all criteria is 
realized by completing six sub-steps. 

The sub steps are listed below in the order they are to 
be accomplished. 

5.1. - Prioritize the overall importance of qualitative 

versus quantitative criteria. 

5.2. - Select applicable computer categories. Each 

category is evaluated separately. 

5.3. - Select criteria for each computer category. 

5.4. - Select sub-criteria for each criterion until 

there are no more sub-criteria. 

5.5. - Prioritize and weight all categories, criteria, 


and sub-criteria. 
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5.6. - Calculate the absolute weights for all criteria 

and sub-criteria. 
In Step 7, the process of receiving, comparing and 
analyzing bids is comprised of two sub-steps: 
7.1. - Assign each relevant model of computer from a 
proposed computer-family to a category 
according to established criteria. 

7.2. - Evaluate each computer-family in accordance with 
the criteria established in Step 5. 

The advantage in applying the AHP to this methodology is 
that greater objectivity is achieved. No longer will an 
individual, or group, have to compare a large number of 
criteria and rank them, including the assignment of a 
relative value for each object. All comparisons are done 
one to one. The ranking and weighting of the criteria is 


computable and replicable. 


ue 
= 


Identification of possible vendors and 
manufacturers. 


Primary elimination of irrelevant candidates. 
Determination of mandatory requirements. 


Examination of vendors’ compliance with 
mandatory requirements. 


Setting quantitative and qualitative criteria 
and respective scales. 


Prioritize overall importance of qualitative 
and quantitative criteria. 


Select applicable computer categories. Each 
category is evaluated separately. 


Select criteria for each computer category 


Select sub-criteria for each criterion until 
there are no more sub-criteria. 


Prioritize and weight all categories, criteria 
and sub-criteria. 


Calculate the absolute weights for all criteria 
and sub-criteria. 


Writing the RFP to be addressed to selected 
vendors. 


Receiving, comparing, and analyzing bids. 

Assign each relevant model of computer from a 
proposed computer-family to a category according 
to predetermined criteria. 


Evaluate each computer-family in accordance 
with criteria established in Step 5. 


Drawing up a final list of vendors. 


Performance of hardware and software benchmarks. 





Drawing final conclusions and selection of best 


Figure 6. Updated workflow diagram. 
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VI. DEVELOPING SELECTION CRITERIA AND SCALE 


A. SELECTING AND PRIORITIZING CRITERIA 

In order to select a computer-family that best fulfills 
its requirements, an organization must designate the 
qualities that will be used to compare the computer- 
families. These qualities, or characteristics, are called 
selection criteria. 

Criteria that are used to evaluate computer-families are 
either quantifiable or non-quantifiable. The qualities that 
are not quantifiable are referred to as qualitative criteria 
and are evaluated subjectively. Those characteristics that 
are quantifiable and measurable by an established standard 
are called quantitative criteria, and are evaluated 
objectively. 

Criteria used in the evaluation process will be placed 
at different levels in a hierarchy. At the top of the 
hierarchy is the "Total Evaluation of a Computer-Family". 
The second level consists of the division of qualitative and 
quantitative criteria. The next level is the first group of 


selectible criteria, in this case categories of computers. 
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At the lowest level are criteria that are elements of a 
criterion at the next higher level in the hierarchy. The 
criteria at each level, which are the descriptors of a 
criterion of the next higher level, are ranked in order of 
importance and scaled by their relative weight. This is the 
process of prioritizing. 

The process of prioritization will be conducted using 
the Analytical Hierarchy Process developed by Saaty (1980). 
This uses a one to one comparison of all the items at the 
same level, producing a relative ranking and weighting of 
each criterion at that level in the hierarchy. These are 
the actions that take place during Step 5 [Fig. 6]. 

Step 5.1. Prioritize the overall importance of 
qualitative versus quantitative criteria. Because there are 
both qualitative and quantitative criteria used in the 
evaluation process, prioritization begins with the 
determination of the weights, or percentage of the total 
score, for each of these groups of criteria. This is a 
subjective decision made without the use of the AHP. It is 


the second level in the hierarchy. 


Step 5.2. Select applicable computer categories. Each 


category is evaluated separately. Prioritization continues 
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with the selection of the applicable computer categories for 
the computer-family and the determination of the relative 
weight of each category. Computer category is the third 
level in the hierarchy. The categories from which to choose 
are: 

¢ Mainframe computer 

e Super-minicomputer 

¢ Minicomputer 

e Super-microcomputer 

¢ Microcomputer 

Step 5.3. Select criteria for each computer category. 

For each computer category designated in Step 5.2, choose 
the appropriate main categories of criteria that can be used 
in the evaluation process. The main categories of criteria, 
taken from the list in Figure 2, are: 

e Hardware 

e Software 

¢ Communication 

e Conversion 

e Environment 
These categories of criteria are the fourth level in the 


hierarchy. 
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Step 5.4. Select sub-criteria for each criterion until 





there are no more sub-criteria. For each main category of 
criteria, Step 5.3, in each main category of computer, Step 
5.2, select the appropriate sub-criteria. If a criterion 
can be further described by sub-criteria, continue the 
process of subdivision. Subdivision may occur down three, 
four or even five more levels. 

In selecting the quantifiable criteria, care must be 
taken to select relevant items and to not continue 
subdividing a criterion just because it is possible to do 
so. The criteria selected should be for valid items of 
comparison so that when the evaluation of the computer- 
families is made, and there is a difference in score between 
computer-families, the difference is valid and meaningful. 

A detailed list of quantitative criteria, which starts 
with the main categories of criteria at Level 4 in the 
hierarchy, is shown in Figure 8. For criterion that can be 
subdivided, the sub-criteria are listed in hierarchy Level 
5, Level 6, and Level 7. 

The qualitative criteria, even though they are evaluated 
much more subjectively, can also be prioritized using 


Saaty’s procedures. A detailed list of qualitative criteria 
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that may be used in comparison of computer-families is also 
shown in Figure 8. These criteria are at level 3 and level 
4 in the qualitative side of the hierarchy. The hierarchy 


is shown in Figure 7. 
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Level 4 Level 5 


Hardware 
Memory 


Disks 


Diskette 
Readers 


Terminals 


Printers 


Backup 


Level 6 Level 7 
Main 
Capacity 
Expandability 
Speed 
RAM 
Capacity 
Expandability 
Number of 
drives 
Capacity 
Size 
Number 
Density 
Monitors 
Monochrome 
Color 
Graphics abilities 
(EGA, EGA, VGA) 
Keyboards 
Voice 
Color 
Graphics 
Laser 
Size 
Size 
Speed 


Figure 8.1. Detailed List of Criteria, Hardware 
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Level 4 


Software 


Level 5 Level 6 
System 
software 
Operating system uniformity 
Utilities 
Compilers 
Compatibility 
Ability to modify 
Documentation 
Programming 
languages 
Compilers 
Interpreters 
Adherence to standards 
Options and extensions 
Development 
tools 
Application generators 
Report generators 
Full screen editors 
Debugging aids 
Macro facilities 
Additional 
software 
Word processors 
Spreadsheets 
Database Management Systems 
E-Mail 
Software 
transferability 


Figure 8.2. Detailed 


List of Criteria, Software 
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Level 4 Level 5 Level 6 


Communication 
Communication 
protocols 


Local area 
networks 
Capacity (Number of 
terminals) 
Maximum distance 
Transmission medium 
Speed 


Wide area 
networks 
Transmission medium 
Internet capability 


Conversion 
Integration 
of current 
programs 


Integration 
of data 
files 


Environment 
New building 
requirements 


Special 
conditions 
Temperature 
Humidity 


Figure 8.3. Detailed List of Criteria, Communication, 
Conversion, and Environment. 
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Level 3 Level 4 


Vendor 

Support 
Training 
Maintenance 
Philosophy 
Practice 

Vendor’s 

reputation 


User opinion 
Trade journal evaluations 


Manufacturer's 
reputation 


User opinion 
Trade journal evaluation 


Number of users 
of computer-family 


Software companies 
specializing in 
this computer-family 


Figure 8.4. Detailed List of Qualitative Criteria 
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Step 5.5. Prioritize and weight all categories, 


criteria and sub-criteria. For each category of computer, 
the appropriate criteria and sub-criteria are listed. After 
all the criteria are listed, they, along with the computer 
categories are prioritized and weighted using Saaty’s 
eigenvector method. [Fig. 9] 

As described earlier, Saaty’s method requires a pairwise 
comparison of those criteria that are located at the same 
level in the hierarchy. Every criterion being compared is 
rated against every other criterion in the same group. A 
value is obtained, based on the values from Figure 5, and 
inserted into the matrix. The size of the matrix should be 
limited to 7 +/- 2 items (Saaty, 1982). The matrix 
generates a relative weight that is given to each category 
or criterion. This is an eigenvector; it has a decimal 
value or percentage of 1. The total value of all the 
weights generated for each group being compared is 1 or 


100%. 


B. PRIORITIZING AND WEIGHTING: A DETAILED EXAMPLE 
After prioritizing the collective qualitative and 


quantitative criteria, Step 5.1, and then selecting the 
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applicable computer categories and appropriate criteria and 
sub-criteria, Steps 5.2 - 5.4, a decision maker is ready to 
begin computing priorities. 

In this example, three categories of computers, selected 
in Step 5.2, are placed in a matrix [Fig 9]. The three 
categories selected are Mainframe computer, Minicomputer, 
and Microcomputer. To determine the relative weight for 
each category, a decision maker has to compare each category 
to the other two categories, one at a time, and select the 
number 1 to 9 which best represents the intensity of 
importance of one category to the other. The scale, shown in 
full in Figure 5, is presented here, in its abbreviated 


form. 


Intensity of 


Importance Definition 
1 Equal importance. 
3 Weak importance of one over another. 
3) Essential or strong importance. 
7 Demonstrated importance. 
9 Absolute importance. 
2,4, 676 Intermediate values between the two 


adjacent judgments. 


The numbers that are inserted into the matrix compare 
the item on the left with the item in the top row and 
represent the value from the scale above. If the item on 
the top is considered more important than the item on the 
left, a fraction, the reciprocal value of the comparison, is 
entered into the matrix. 

Sample values have been entered into the matrix. 

(Fig. 9.1] Using the sample values an eigenvector will be 
computed for each computer category. The eigenvector will 


be used as the relative weight of the computer category. 


Hierarchy Level 3 


| 
| 
1 Main- | 


| 

| 

| 

| Computer Category | frame Mini Micro 

| Mainframe | il 5 3 | 
| Mini | 1/5 i 1/4 | 
| Micro 1/3 4 1 | 


(a i I ESS SS SSS 
Figure 9.1 The Initial Matrix 


The matrix is then normalized by adding the values in 
each column, then dividing each entry in each column by the 


total of that column. A new matrix with normalized values 


is created. [Fig. 9.2] 


| Main | 
lcomputer Category | Frame Mini Micro | 
| Mainframe | 1 5 3 | 
| Mini Lass 1 1/4 | 
| Micro | 173 4 aL | 
| c= a= = 

Icolumn total 1.533 10 4.25 | 


| Main | 


| computer Category | Frame Mini Micro | 
| Mainframe | 0.652 0.500 0.706 | 
| ming | 0.131 0.100 0.059 | 
| Micro | Os257 omane 0.235" 
| | 


EEE ee el 


Figure 9.2 The Normalized Matrix 


The last step is to average over the rows by adding the 
values in each row of the normalized matrix and dividing the 
rows by the number of entries in each. This number is the 
eigenvalue and is the weight that is given to that category. 


[Fig. 9.3] 


<a | 


| Computed | 
| Relative | 
| Weight | 
| 65cm oO! + V706 | 
| Mainframe go — seeeesss-n-e-------- = .6193 | 
| 3 | 
| | 
| | 
| Pelee en a ae ee | 
PMini ween nn nn nnn ------- = .0967 | 
| 3 | 
| | 
| .217 + .400 + .235 | 
ee  - ek. eel 
|  ———— | 
| 1.0000 | 


Figure 9.3 Computing the Eigenvalue 


The total relative weight for all computer categories is 
equal to 1 or 100%. 

Consistency of the values must be taken into account in 
computing the eigenvectors using the calculated consistency 
index (CI), dividing the CI by the random value of 
consistency from a table. The random value is dependent on 
the number of items in the matrix. The resultant is the 
consistency ratio (CR). The CR should be 10% or less, 
indicating judgments in comparing items in the matrix were 


not randomly made. (Saaty, 1982) 
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The matrices listed in Appendix A have listed in them 
criteria and sub-criteria from hierarchy Level 4 down 
through Level 7. For this example, only the first criterion 
from the previous level will have its sub-criteria listed in 
the next lower level. The numbers along the top row of each 
matrix refer to the corresponding numbered item on the left 
side of the matrix. The weight, in the last column, is the 
relative weight, the eigenvalue, of the criterion named in 
that row, in the left column. 

For this example, the Mainframe Computer category is 
used. Its relative weight has already been calculated. 
[Fig.9.3] The main categories of criteria are selected, and 
relative weights are computed for each of these criteria. 
The first matrix determines the relative weights a decision 
maker places on each of the main categories of criteria for 
each of the computer categories. The decisions made here 
have an effect on the importance of the decisions made lower 
in the hierarchy. For example, if hardware turns out to be 
worth 75% in the evaluation of minicomputers, the remaining 
categories of criteria are worth only 25%, no matter how 
many sub-criteria are used in the evaluation process. This 


is another reason not to subdivide criteria unnecessarily. 
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Step 5.6. Calculate the absolute weights for all 


criteria and sub-criteria. Following the method proposed by 
Borovits and Zviran (1987), a scoring system that makes 100 
points the maximum score attainable for a computer-family, 
will be used in conjunction with the AHP. 

The absolute weight for a criterion is computed by 
multiplying its relative weight by the relative weight of 
each of its predecessors in the hierarchy [Fig. 10], or by 
the absolute weight of its immediate predecessor. When a 
computer-family is evaluated the maximum score available for 
any criterion is its absolute weight. A list of absolute 
weights will be shown on an evaluation sheet, Appendix B 
used in conjunction with the scoring of the competing 
computer-families, in Step 7. 

An example of how an absolute weight is calculated is 
shown in Figure 10. A sample value is used for the relative 
weight of quantitative criteria. The relative weight for 
computer categories, mainframe computer was computed in 
Figure 9. 

The process is demonstrated in Figure 10 and is 
completed for each criterion in the hierarchy of a computer- 


family. The absolute weight for the criteria whose relative 
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weights were calculated in Appendix A are shown on the 
Vendor Evaluation Form in Appendix B. 


Os | 


| Computer- Quantitative Mainframe 
Family Criteria Computer 

lrotal Weight Relative Weight Relative Weight 
| 

| = 1.00 = 0.80 = 0.6913 


lAbsolute Weight for Mainframe Computer = 
[(1.00) x (0.80) x (0.6913) = 0.4954 


Figure 10. Computing an absolute weight of a 
criterion 
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VII. THE EVALUATION PROCESS 


After the process of prioritizing the criteria has been 
completed, the process of evaluating each of the competing 
computer-families begins. In Borovits and Zviran’s (1987) 
methodology this is accomplished as part of Step 7. 

In response to the request for proposal (RFP) that was 
sent out to vendors, bids for proposed computer-families 
will have been received. These bids then have to be 
analyzed and evaluated with scores assigned. In Steps 8, 9, 
and 10, the highest rated computer-families, on the basis of 
score, are selected for further evaluation and testing and, 
ultimately, one will be selected as the winner. 

The scope of Step 7 is the initial evaluation and 
scoring of the proposed computer-families, submitted in 
response to the RFP. As stated previously, this is 


completed by two sub-steps. 
Step 7.1. Assign each relevant model from _a proposed 


computer-family to a category, according to predetermined 


criteria. In Step 5.2, computer categories were selected. 


The decision maker chose the computer categories that met an 
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organization’s needs. Each relevant computer model from the 
proposed family will be placed in the appropriate computer 
category and will be evaluated by the criteria already 
selected for that category. 

The determination as to which category a computer will 
be placed is based on, but not limited to, such factors as 
CPU, memory or disk drives. The decision as to what factors 
will constitute placement into a particular category will 
have been determined when the categories were selected in 


Step 5.2. 


Step 7.2. Evaluate each computer-family in accordance 


with the criteria established in Step 5. Using the 


evaluation form, Appendix B, each computer-family is 
evaluated, criterion by criterion, with the result posted on 
the form. The total for the computer-family is then 
computed and a final score is established. 

The maximum score possible for any criterion is its 
absolute weight. The score assigned and posted on the form 
is on a scale of 0-1. Complying with the criterion 
completely is scored as 1, anything less is a decimal value 
less than 1. The posted score is multiplied by the absolute 


weight to yield the absolute score for that criterion. All 


a2 


of the absolute scores are added to yield the final score 
for the computer-family. Only the lowest level criteria are 
scored. The maximum total score is 100. When this process 
has been completed for each of the computer-families, an 
objective comparison based on the final score may be made. 
The three or four highest rated computer-families may then 
be evaluated further by operational and benchmark testing. 
In Appendix B, the relative weights computed in Appendix 
A have been inserted into the evaluation sheet. Using those 
relative weights, the absolute weights for all criteria 


cited in the example have been computed. 


33 


VIII. CONCLUSION 


The process of selecting a computer-family is a complex 
procedure. The goal for a decision maker, responsible for 
selecting a computer-family, is to select the correct line 
of products for an organization. Because of the complexity 
of the selection process, a formalized methodology makes the 
process more objective. 

Using the methodology presented here, along with 
methodology developed by Borovits and Zviran (1987), an 
objective selection process is created. The process allows 
the designation of appropriate criteria to be used in the 
selection of a computer-family. The appropriate criteria 
are those selected by an organization trying to select a 
computer-family, based on an evaluation of its needs. 

Once criteria have been selected, they are objectively 
prioritized and weighted, establishing their net value and 
absolute weight in an overall evaluation of a computer- 
family. Each computer-family is evaluated and scored 
separately, but in accordance with the prioritized and 


weighted criteria. 
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In spite of the careful process of prioritizing and 
weighting, total objectivity can not be achieved. 
Individuals comparing criteria will still be making 
subjective judgments as to the relative importance of one 
criterion versus another. This cannot be avoided. 

Subjectivity in the comparison of criteria may be 
counteracted by checking for consistency. The ability to 
measure the consistency in the comparison of criteria adds a 
check and balance to the process not previously present. 

By following the procedures developed in this thesis, 
the process of selecting a computer-family is made reliable 
and objective. The end product of this process, a computer- 
family that best meets the needs of an organization, may be 
chosen with the knowledge that the correct computer-family 


was selected. 
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APPENDIX A 


Sample values, using the guidelines from Figure 5, have 
been inserted into the matrices. Using the process in Step 
5.5, each matrix was normalized. The relative weight listed 
in the last column is the eigenvalue of the criterion in the 


first column in that same row. 


Level 3 


Relative 
Computer Category Weight 


1.Mainframe 0.6193 
2.Mini 0.0964 
3.Micro 0.2842 


1.0000 


Main Categories Relative 
of Criteria (Mainframe) Weight 


1.Hardware 0.3024 
2.Software 0.4119 
3.Communication 0.0775 
4.Conversion 0.1656 
5.Environment 


1.0000 
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Relative 
Hardware Weight 


1.Memory 0.3633 
2.Disks M3 1 0.1663 
3.Diskette Readers 1/6 1/3 0.0420 
4.Terminals w2.3 0.2631 
5.Printers 279 73 0.0986 
6.Backup 174 1/5 0.0667 


Relative 
Weight 


Level 7 
Relative 
Main Memory Weight 


1.Capacity 0.1593 

2.Expandibility 0.2518 

3.Speed 0.5889 
1.0000 
Relative 


Weight 


1.Capacity 
2.Expandibility 





APPENDIX B 


Vendor Evaluation Sheet Relative Absolute Absolute 
Weight Weight Score Score 


ocan ft 00]-000f | 
lguantitative | 000] 8000] 

Mainframe | 93.4954] 

Hardware 302 nage] | 
Memory | 3693] -o5aa] | 
main 2500] one] | 
capacity a9] .o022] | 
Expanaibitity —_|.2528|.0034] | 
speeds] 0080] | 
paw 7500] 0408] | 
capacity | 2000] 0082] | 
Expanaivitity | -e000|.0326| | 
ES ee) ee 
umber of drives | | | | 
capacity | TT 
isxette Readers | .0420|.0063| | 
Pas 
momer TL 
ave ais 
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Vendor Evaluation Sheet Relative Absolute Absolute 
Mainframe Hardware (cont’d) Weight Weight Score Score 


frerminars 2630] one] | 
goto ee 
Monochrome | | || 
a Ae ee 
ccapnics apitities | | | | 
Keyooarts | || 
vei 
printers | 006] .onae| | 
ow 7 eS ee 
2 ee 
ao ES 
stay i 
[pace 067] 0020] | 
on Se oe 
a eS Ee ee 
soreware azo} zon] 
[system software | ||| 
lop. aystem unisormity | | || 
oa SS | 
comiters | || 
compatapirsey | || 
apiaity to moaity | ||| 
pocumentation | | | T 




































oy 


Vendor Evaluation Sheet Relative Absolute Absolute 
Mainframe Software (cont’d) Weight Weight Score Score 


Progranming languages | | || 
comitese | || 
ee 
Adnerence to standards | | | | 
loptions and extensions | | | | 
peveropment coors | ||| 
application generators | | | | 
Report generators | | | | 
rut acreen eaitors | | | | 
Depugging aias =| TT 
Macro facinities ||| 
Additional software | | || 
Word processors | | || 
spreaasnects | | | 
Database mgmt. systems | | | | 
ERE 
software transterapitity | | | | 
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Vendor Evaluation Sheet Relative Absolute Absolute 
Mainframe (cont’d) Weight Weight Score Score 


communication | 0775] 030] | 
communication protocons | | | | 
fuocat area networks | | || 
capacity (# of terminate) | | | | 
Maximun distance | | || 
teananicaion medion | | || 
po SS Se Sa 
wide azea networks | ||| 
teananiesion medium | | | | 
internet capabirity | | || 
conversion | 2656) .0822| | 
integrate current progs.| | | || 
[integration of data sites! | | || 
Eovironment | oaze|-ozaa] | 
New building requirements] | | || 
[special conditions | | || 
memperature | || 
womsaity | TT 
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